Search for: All records

The BigCode community, an open-scientific collaboration working on the responsible development of Large Language Models for Code (Code LLMs), introduces StarCoder and StarCoderBase: 15.5B parameter models with 8K context length, infilling capabilities and fast large-batch inference enabled by multi-query attention. StarCoderBase is trained on 1 trillion tokens sourced from The Stack, a large collection of permissively licensed GitHub repositories with inspection tools and an opt-out process. We fine-tuned StarCoderBase on 35B Python tokens, resulting in the creation of StarCoder. We perform the most comprehensive evaluation of Code LLMs to date and show that StarCoderBase outperforms every open Code LLM that supports multiple programming languages and matches or outperforms the OpenAI code-cushman-001 model. Furthermore, StarCoder outperforms every model that is fine-tuned on Python, can be prompted to achieve 40% pass@1 on HumanEval, and still retains its performance on other programming languages. We take several important steps towards a safe open-access model release, including an improved PII redaction pipeline and a novel attribution tracing tool, and make the StarCoder models publicly available under a more commercially viable version of the Open Responsible AI Model license. 

For brittle friction and rock deformation, the coefficientαin the general effective stress relationσ^e = σ − αP_pcan be approximated as unity with sufficient accuracy. However, it is uncertain ifαdeviates from unity for semibrittle flow when both brittle and intracrystalline‐plastic deformation is involved. We conducted triaxial and isostatic compression experiments on synthetic salt‐rocks (∼300 ppm water) at room temperature to test the effective stress relation in the semibrittle regime using silicone oil and argon gas as pore fluids. Confining and pore pressures were cycled while their difference (differential pressure) was kept constant, such that changes in the mechanical behavior would indicate deviation ofαfrom unity. Microstructural observations were used to determine the dependence ofαon true area of grain contact from asperity yielding. In triaxial compression experiments, semibrittle flow involves grain boundary cracking and sliding, and intragranular dislocation glide and cracking. Flow strength remains constant for changes in pore fluid pressure of more than two orders of magnitude. In isostatic compression experiments, samples show combined processes of microcracking, grain boundary sliding, dislocation glide, and fluid‐assisted grain boundary migration recrystallization. Volumetric strain depends directly on the differential pressures (i.e.,αequals one). Analysis of grain‐contact area in both experiments indicates thatαis independent of the true area of contact defined by plastic yielding at grain boundaries. The observation ofαeffectively equals one may be explained by operation of pressure‐independent intracrystalline‐plastic mechanisms and transmission of pore pressure at grain boundaries through thin fluid films.